A kernel interface is a low-level communication bridge between a process and the kernel of an operating system. The kernel interface typically includes entry points where a process' execution enters the kernel and calling conventions, which control how function arguments are passed to the kernel and how return values are retrieved.
The kernel interface of a typical operating system may include functions to start and stop threads of execution, to synchronize threads of execution, to read and write data from files, to enlist in regions of shared memory, to draw on a screen, to communicate with a network, or to read a system clock. Kernel interfaces can range from dozens of entry points to hundreds and even thousands of entry points.
A kernel interface is one component of the operating system's application programming interface (API) presented to programmers. An API typically defines the interface between an application programmer's source code and operating system libraries so that the same source code will compile on any system supporting the API. Many conventional operating systems encapsulate the kernel interface within a higher-level API because the kernel interface varies from version to version of the kernel.
For example, to insulate application programmers from changes in the kernel of the Microsoft® Windows® XP brand operating system, the kernel interface of that operating system is hidden behind the Win32® API and not published. In the case of the Microsoft® Windows® XP brand operating system, programs are dynamically linked to the libraries of the Win32® API at load time.
On the other hand, a conventional kernel interface allows processor-executable instructions to function without changes on any system using the same or compatible kernel interface. From a security perspective, the processor-executable instructions of the conventional kernel interface are secure from tampering as they reside within the protection domain of the operating system kernel. On the other hand, the processor-executable instructions of the conventional high-level API may not be secure from tampering as they execute within the context of the process.
A common characteristic of conventional operating system kernel interfaces is that they use an access control pattern to decide which operations may be performed by a process through the kernel interface. With an access control pattern each “kernel object”—such as a thread, open file, or user interface windows—has an access control list that specifies which security principals are allowed to perform a specific action (or set of actions) on the kernel object. Kernel objects are abstractions provided by the kernel. Kernel objects are the things that an application affects through the kernel interface. Examples of kernel objects in the Microsoft® Windows® XP brand OS are “open files”, “windows”, “threads of execution”, “shared memory mappings”, etc.
Security principals (i.e., “security ids”) are the unit of identifying a party for security purposes. For example, on Unix, each user is a security principal with their own unique user id. Security principals are not always users. For example, the Microsoft® Windows® XP brand OS has a “network logon” principal that is used to run services that communicate over the network. As security principals typically have a one-to-one correspondence with users, each process typically has one associated security principal.
Metaphorically, an access control list is like guest list at private gathering, only processes on the list are allowed to use the kernel object. Because a running kernel may have thousands or tens of thousands of objects—such as threads, open files, and user interface windows—and dozens or hundreds of security principals, the access control lists of kernel can become unmanageable. Incorrectly set access control lists on kernel objects are a common source of security vulnerabilities in conventional operating systems.
To avoid the complexities of managing access control lists, other operating systems employ a capability pattern to control which operations may be performed by a process through the kernel interface. With a capability pattern, a “capability” represents the right to perform a specific action on a specific kernel object. The capability can be shared from one process to another to grant access to the kernel object. Metaphorically, a capability is like a key to a door, any process holding a capability can access the object that the capability unlocks. The complexities of tracking potentially hundreds of capabilities within a single process can quickly lead to security vulnerabilities in which a capability is accidentally leaked to another process. Furthermore, like a key, it is extremely difficult to get back a key once it has been lent out or lost.